Depth inference is a key research area for modeling 3D objects in the 3D environment; for consumer electronics, robotics, and computer vision. In consumer electronics, depth maps are used in Depth Image Based Rendering (DIBR) displays, they are used as part of improved efficiency 3D compression algorithms, and can be used in future virtual reality. Depth may be inferred using stereo disparity; however this requires multiple source images where two cameras or complex optics are needed to achieve the left-right views. Depth also may be found by ranging techniques, but this requires additional transmit and receive hardware. New light-field or integral imaging cameras can produce depth, but the microlens array reduces the maximum imager resolution capability. Typical current 3D imaging systems cannot be easily miniaturized to fit with the form factor of a small consumer camera, such as the type in cell phones and tablet devices. In addition, the cost of the current systems includes two imagers and/or expensive lens arrays or ranging devices.
Depth from defocus inference requires only one imager capturing two focus images, which can be done with a standard camera with varying focus. Inferring depth is done by a pixel-by-pixel comparison of two or more defocussed images, where the object's blur radius is related to its distance. This depth inference uses Bayesian and Markov Random Field (MRF) statistical structure. The classical approach can be improved by combination with other computational imaging techniques.
Extended Depth from Defocus (EDfD) extends classical DfD using a new optimization function, extended to adapt to both the image's color data and high frequency image data. Significant depth accuracy improvements compared to the currently published DfD techniques have been observed.
Depth is important in new consumer electronics products in order to create immersive 3D experiences for the user with new 3D displays. Accurate depth information is also needed for improved compression efficiency and for super-resolution techniques. A method for enhancing a ranging camera's resolution has been reported using Markov Random Field methods with the 2D image to provide a more accurate depth result for DIBR display, in which a ranging camera is used in addition to the visible light imager. Additional methods of 2D to 3D conversion using edge information from the 2D image to provide a depth map from a hypothesis depth map starting point; or by providing a depth map specifically for outdoor scenes using the dark channel (the effect of haze in the image) to estimate depth. The results from EDfD show significant quality improvement compared to these two papers, and EDfD is generally applicable to a variety of scenes.
For the EDfD method, fast focus optics is required. Bio-inspired microfluidic lenses allow a time-domain approach for the very fast focus change. These lenses use two fluids and electrostatic forces to rapidly change the shape of a very small lens. System design requires balancing the maximum focus speed of the microfluidic lens with the capability and accuracy of the depth inference.
An extended DfD depth inference method, together with a fast focus lens which enables depth map generation of an average accuracy 4.67 RMSE compared to ground truth, and small size due to a single imager is presented. The computational complexity is similar to other methods. The results are provided for synthetic blur images for accuracy testing and for a single imager matched with microfluidic lens for generating the 2 focus images.
Corresponding reference characters indicate corresponding parts throughout the several views. Although the drawings represent embodiments of the present invention, the drawings are not necessarily to scale and certain features may be exaggerated in order to better illustrate and explain the present invention. The exemplification set out herein illustrates an embodiment of the invention, in one form, and such exemplifications are not to be construed as limiting the scope of the invention in any manner.